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Acceleration of relaxation toward a fixed stationary distribution via violation of detailed balance 
was reported in the context of a Markov chain Monte Carlo method recently. Inspired by this 
result, systematic methods to violate detailed balance in Langevin dynamics were formulated by 
using exponential and rotational nonconservative forces. In the present paper, we accentuate that 
such specific nonconservative forces relate to the large deviation of total heat in an equilibrium 
state. The response to these nonconservative forces can be described by the intrinsic fluctuation 
of the total heat in the equilibrium state. Consequently, the fluctuation-dissipation relation for 
nonequilibrium steady states is derived without recourse to a linear response approximation. 


I. INTRODUCTION 

One of the focal topics in nonequilibrium thermody¬ 
namics and statistical physics is an extension of the 
fluctuation-dissipation theorem (FDT) [jj. The well- 
known FDT for equilibrium states owes its success to 
two relations: a fluctuation-response relation and a re¬ 
lation between the response and energy dissipation. The 
fluctuation-response relation claims that the response of 
a macroscopic quantity to a perturbation is given by its 
intrinsic fluctuation under the unperturbed dynamics. 
On the other hand, for near-equilibrium state, a linear 
response to a small perturbation can be expected. The 
energy dissipation is hence given in terms of the response. 
In far-from-equilibrium states, however, a general rela¬ 
tion between energy dissipation and the response is miss¬ 
ing, while several extensions of the fluctuation-response 
relation are known 09- Only in the linear response 
regime near nonequilibrium steady states (NESS), where 
the energy dissipation can be macroscopically evaluated, 
has the genuine fluctuation-dissipation relation been ob¬ 
tained 

Highlighting a series of Onsager’s works [?,, Q, 
Onsager’s regression hypothesis gives the fluctuation- 
response relation in near-equilibrium states. Onsager’s 
regression hypothesis assumes that the correlation func¬ 
tion in equilibrium state ( intrinsic fluctuation) governs 
the relaxation from near-equilibrium state toward equi¬ 
librium that is given as the response to an appropriate im¬ 
pulse perturbation. On the other hand, Onsager’s prin¬ 
ciple provides a relationship between response to an ex¬ 
ternal perturbation sustaining a steady state and energy 
dissipation in the near-equilibrium steady state, which is 
housekeeping heat characterizing violation of the detailed 
balance condition (DBC). Onsager’s principle states that 
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the current as a response is proportional to the exter¬ 
nal perturbation called thermodynamic force. Thus the 
housekeeping heat that is given by a product of the cur¬ 
rent and its conjugate thermodynamic force is always 
non-negative. Combining Onsager’s regression hypoth¬ 
esis (fluctuation-response relation) and Onsager’s prin¬ 
ciple (relation between response and dissipation) yields 
FDT in near-equilibrium state. Recently, the influence 
of the violation of the DBC on relaxation has been dis¬ 
cussed in the context of a Markov chain Monte Carlo 
method (MCMC) It is guaranteed in terms of eigen¬ 
values for a transition matrix that the relaxation toward 
a fixed stationary distribution is accelerated by the vio¬ 
lation of the DBC Q. Furthermore, it has been found 
that applying exponential and/or rotational forces as a 
systematic method to violate the DBC accelerates relax¬ 
ation in Langevin dynamics mm- In these works on 
MCMC, the energy dissipation is obviously given by a 
housekeeping heat, which is a product of probability cur¬ 
rent and nonconservative force violating DBC. If FDT in 
NESS far from equilibrium is expected, relaxation (re¬ 
sponse) should be connected with intrinsic fluctuation. 
However, what is the intrinsic fluctuation relating the re¬ 
laxation or response in this case? In the present paper, 
we give a possible answer to this question. 

The main result of this paper is to give the relation 
between the large deviation function for total heat in 
an equilibrium state and the expectation of that in the 
NESS. The large deviation function describes the intrin¬ 
sic fluctuation in the equilibrium state. In addition, the 
housekeeping heat, which is equivalent to the total heat 
in the NESS, expresses the energy dissipation. Therefore, 
our result is interpreted as the extension of the FDT in 
the NESS. The result is derived without resorting to a lin¬ 
ear response approximation as used in several extensions 
of the FDT in the NESS. In other words, our extension 
leads to a full-order form of the FDT in the NESS. The 
fluctuation-response relation for the total heat is derived 
through the framework of the Nemoto-Sasa theory [Il¬ 
ia , which gives the relation between the large deviation 



2 


function for intrinsic fluctuations and response to appro¬ 
priate perturbations. The response part of this relation 
contains the housekeeping heat emerging from a topolog¬ 
ical argument, which is the general framework given by 
Sagawa and Hayakawa fl5j |. 


II. SET UP 

In the present paper, we deal with the two systems, 
which are related by the additional force u on the N 
degrees of freedom. We refer the case without any addi¬ 
tional force as the original system and that with nontriv¬ 
ial u as the biased system. The system is governed by 
the following Langevin dynamics 

dx(f) = [A (x(i)) + u (x(f))] dt + \/2TdW(t), (1) 

where A is the drift term for the original system, W (t) 
is the standard Wienner process, T denotes the noise 
intensity or temperature, and a midpoint prescription 
x(f) = [x(t + dt) + x(f)] /2 is used for Stratonovicli in¬ 
terpretation of stochastic dynamics. 

The housekeeping heat for the dynamics © is defined 
as UK 

Qhk = f (A + u - TgradlnP") o x(t)dt, (2) 

Jo 

where P“ is the stationary distribution and o stands for 
the multiplication in the sense of the Stratonovich. On 
the other hand, in NESS, the excess heat is defined as 

Qex = —TlnP" (x(t)) + TlnP" (x(0)). (3) 

Using these quantities, the total heat is defined as Q tot = 
Qhk + Qex- Since (A + u — Pgradln P") P" is the proba¬ 
bility current, which characterizes the violation of DBC, 
the housekeeping heat Qhk vanishes if the DBC is satis¬ 
fied. 


III. NEMOTO-SASA THEORY AND 
VARIATIONAL PRINCIPLE 


For convenience, we briefly review the formulation of 
the Nemoto-Sasa theory from the viewpoint of a varia¬ 
tional principle 0,111. The Nemoto-Sasa theory origi¬ 
nally gives the relation between the cumulant generating 
function of a current in the NESS of the original system 
and an expectation of the current in the biased system 
0]. The conditional path probability for a path real¬ 
ization X with an initial condition x(0) = Xq is given 
as 


L u (X|xo) 

exp ( ~ ir J 0 dt ^ ~ A - u (aW )] 2 

~IJ Q (*(*))+ u (*(*))] }• 


Note that 


midi H = ±f T dt 


L„(X|x 0 ) 2 T J„ 

1 

~~ 2 


u ■ (A + u) — (2A + u) ■ — 


dt divu 


u 


V2T 


AT 


-dt — 


divu 


dt -f- 


V2T 


odW 


orfW(i). (5) 


Since the expectation of the second term in the last line of 
Eq. © is canceled by that of the third term, we find the 
scaled Kullback-Leibler (KL) divergence between path 
probabilities with and without the additional force u as 


D[L u \L 0 ] 


/I T u (X|x 0 ) 
Aoo\t P 0 (X|x 0 ) 


dX ~~4T^ Ps *( X ) 



( 6 ) 


where (-) u denotes the ensemble average under the 
stochastic dynamics ©■ To investigate the full-order 
cumulant, we recall the scaled cumulant generating func¬ 
tion Ao( 7 ) for an arbitrary time-averaged quantity S'(X) 
of the original system defined as 

A 0 ( 7 ) = lim — In (exp ( 7 r 5 (X ))} 0 . (7) 

T—)• OO 7 " 

Here let us minimize D[L u \Lo] under the constraint that 
the expectation (S(X)) U depending on the path realiza¬ 
tion X is fixed. The scaled cumulant generating function 
then emerges as 

V 7 ) = maxj 7 (S(X)) U - J dx^^P"(x)| , ( 8 ) 

where we set 7 as a Lagrange multiplier 0]. The 
cumulant generating function satisfies the following 
fluctuation-response relation: 

^M = (5(X)) u7 , (9) 

where the special additional force u 7 is given by 

u 7 =argmax jy (S(X)) U - j dx^J^P"(x)j .(10) 


The fluctuation-response relation indicates that the cu¬ 
mulant generating function in the original system can 
be estimated through the measurement of the quantity 
S(X) in the biased system. 

On the other hand, the large deviation function of 
S(X) of the original system 

I 0 (s) = — lim — In [Probo (S(X) = s)], (11) 

T—y OO T 

where Prob 0 (S(X) = s) denotes the probability that 
S(X) = s in the original system, is given by the min¬ 
imum of the KL divergence as 

I 0 (s) = mini) [P U |P 0 ] subject to S(X) = s.(12) 


(4) 
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This relation is immediately obtained by the Legendre 
transformation on Eq. ((SJ). The special additional force 
gives the solution of this equality. In other words, the 
large deviation function of the original system can be 
evaluated through the biased system as shown in Eq. 
©. Substitution of S(X) = f ( J xdt/r into Eqs. (j 8 j) and 
(fTUl) indeed reproduces the Nemoto-Sasa theory for the 
current cumulant generating function [12) . Note that the 
above formulation is valid for an arbitrary quantity S (X), 
not only for the current. 


where Xi and u, are the ith components of x and u, 
respectively, and 7 ij{x\ij) is an arbitrary antisymmetric 
matrix independent of Xi and Xj. Here xw ;/ denotes an 
(N — 2)-dimensional subvector given by the elimination 
of the ?’th and jth components from x. (ii) Exponential 
force is 

^i(x) =7i( x \i)exp[ / 9[/ i (a; i )], (17) 


IV. TWO CHOICES OF DBC VIOLATION IN 
MCMC 


The MCMC is a powerful tool for providing a sequence 
of random numbers following a desired distribution. Var¬ 
ious techniques to make the relaxation to the stationary 
state faster have been proposed [l7l - l2l| . While these con¬ 
ventional MCMCs impose the DBC to ensure the con¬ 
vergence of the system toward the desired distribution, 
the DBC is only the sufficient condition for the conver¬ 
gence. Several algorithms violating the DBC actually 
exhibit high-speed convergences toward the desired dis¬ 
tribution [23 - l25l | . General proof for this acceleration has 
been mathematically provided [§]. Furthermore, a sys¬ 
tematic method to violate the DBC emu is provided 
as follows. 

The desired distribution P ss in the MCMC often takes 
the form of an exponential family, or in physical terminol¬ 
ogy, a Gibbsian distribution. It is wellknown that such a 
distribution is realized as an equilibrium distribution by 
the following Langevin dynamics with the DBC: 

dx(t) = —grad U (x(t)) dt + VzidW(t). (13) 

Here U is a scalar potential given as the summation of 
the ingredients associated with the ith degree of freedom 
Ui(x) as U(x) = JTf7i(x), an d T = 1//3 temperature. 
The equilibrium distribution for this dynamics is given 
as 


P ss (x) = exp {—(3U (x)] /Z, (14) 

where Z is a partition function. In the systematic method 
violating DBC while keeping the Gibbsian distribution 
m as a stationary one, we add a non-conservative force 
u to the dynamics dl3l) : 

dx(t) = —grad U (x(t)) dt + u (x(t)) dt + V2TdW(t). 

(15) 

According to the Fokker-Planck equation corresponding 
to Eq. m we heuristically find two solutions for u to 
keep the Gibbsian distribution (H4l) as a stationary distri¬ 
bution for the modified dynamics [ldt (i) Rotational 
force is given by 


Ui (x) 


v-' , ,dU (x 

z- m x w) dxi 


j«i) 


^( x vj) 

j(>o 


dU(x) 

dxj 


(16) 


where 7 j(x\j) is an arbitrary function independent of the 
ith component Xi. The independence of Xi and Xj in 
the constant 7 y and that of Xi in 7 j in the additional 
force comes from the condition that fixes the stationary 
distribution. For the exponential force m, a periodic 
boundary condition should be imposed due to the proba¬ 
bility conservation, while the original dynamics m can 
be solved under the natural boundary condition. 


V. ROTATIONAL FORCE 


The additional force u violating the DBC was first for¬ 
mulated to accelerate the relaxation to the stationary 
distribution for practical use in numerical computation. 
On the other hand, as shown below, it naturally emerges 
from the optimization of the KL divergence. Let us con¬ 
sider the long-time average of work performed by the 
external force A + u with A = —grad U: 


W u = lim — I [A(x) + u(x)] • dt, (18) 

t->oo T J C 

where the integral is performed on the trajectory C in 
the state space, which is given by the path realization X. 
Supposing that the trajectory C is compact and no source 
exists, W u vanishes for any open trajectories C. This 
assumption is valid if no divergence of the probability 
current exists in the stationary state. Therefore we focus 
on the case only for closed trajectories. Using the Stokes 
theorem, we find 


W 


u 



d [A{, 4 - iii] 

d Xj 


dxi A dxj , 


(19) 


where the integration is carried out in the region D whose 
boundary is C, i.e., C = dD , Njj is the number of rota¬ 
tions in the i-j plane, and A denotes a wedge product. 
Note that N t j depends on the location, but is indepen¬ 
dent of Xi and Xj. Then the time derivative of house¬ 
keeping heat, which coincides with (VF U ) U because of 
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stationarity of the system after long time, is evaluated as 

(Qh k ) u = <TF u ) u 

= - / dx \i,J r v(x\ij) P <*\iA x \ij) 


i<j 

x / dxidxj 


du,i duj 


dxj dxi 


Pssj.j (Xi , Xj ) 
= ^ J j Vij (x\ ii3 - )Pss\i 7 j ) 


i<j 

x / dxidxj 


u i~R -Wj-— 

OXj axi 


Pss,i,j {Xj , Xj ) , 


( 20 ) 


where (xwj) is the rotation rate defined as ry, (xwj) = 
liniT-^oo (JVjj — iVjj) /r. Here we assume that the sta¬ 
tionary distribution P“ is independent of the additional 
force u, i.e., P“ = P ss , since we focus on the convergence 
toward a given distribution. The marginal distributions 
P S s,i,j and P ss \i,j a 1 '® defined as 


periodic boundary condition. Under a periodic bound¬ 
ary condition, in addition to the loop trajectory created 
by the rotational current, winding on the manifold of a 
state space can generate a closed trajectory. Here we fo¬ 
cus on the work performed by the nonconservative force 
u corresponding to the path of such a nontrivial homo- 
topy. For a winding trajectory, the long-time averaged 
work performed by the force —gradP + u is given as 

W u = lim —- (f m(x)dxi, (24) 

r ~> 00 ^ T J c . 


where N, denotes the winding number of the trajectory 
in the ?’th direction. The integral is taken over the single 
loop Ci in the itli direction. Note that Ni is independent 
of xi but it may depend on the location of a winding x\ ; = 
(xi, • • • , Xi- 1 , Xj+i, • • • , Xn) T ■ Then the expectation of 
the time derivative of housekeeping heat is given as 


Q hk ) u = (W u ) u 

= YlJ ^( x \i)wz(x)P ssV ;(x v )dx, (25) 


Pss,i,j {Xj , Xj ) J dl£\ij P ss (x), (21) 

Pss\i,j ^ dxidxj p ss (x). ( 22 ) 

Note that the rotation rate plays the role of a con¬ 
trol parameter, which yields a steady probability cur¬ 
rent in the NESS. By substituting 5(X) = Qhk into 
Eq. (fTOl) with the stationary distribution (fl4l) , the heuris- 
tically found rotational force m is reproduced with 
7 ij(x\ij) = — 27 nj(x\ij). In addition, the KL diver¬ 
gence © f° r the optimized u relates to the time deriva¬ 
tive of the housekeeping heat as 

£>[L u -r|Lo] = ^(Qhk) uT . (23) 

The additional force u 7 derived here emerges from a 
topological argument on the path realization X. Such a 
topological effect in nonequilibrium thermodynamics was 
discussed by Sagawa and Hayakawa [l5[ . It is pointed out 
that the excess entropy production generally depends on 
the path realization X because it is expressed in terms 
of a vector potential. However, the mathematical frame¬ 
work given by Sagawa and Hayakawa is applicable to ar¬ 
bitrary quantities dependent on a path realization, not 
only for the excess entropy production. The housekeep¬ 
ing heat in our case can be regarded as one of such ex¬ 
amples, since it depends on the path realization of the 
closed trajectory C. 


VI. EXPONENTIAL FORCE 


where 77 (x\,) is the winding rate in the zth direction 
defined as rj(x\j) = lim^oo (iVj) u /r, and P ss \i is the 
marginal distribution defined as 

P ssV (x v ) = f P ss (x)dxi. (26) 

Similarly to the case of the rotational force, by substitut¬ 
ing S(X) = Qhk into Eq. (ITOl) . we find 

ul(x) = 2T 1 r. l (x\ l ) Pss ^ i) . (27) 

Therefore the exponential force 113 with 7 i = 2Tjr z P ss \i 
is reproduced from the Nemoto-Sasa theory. In addition, 
the above obtained exponential force satisfies the relation 
(12511 . the same as the case of the rotational force. We 
emphasize here that our proposed exponential force is 
related to a nontrivial homotopy beyond that described 
in Ref. Il5| . The argument on an entropy production in 
Ref. [13 neglected the effect of boundaries. 


VII. FDT AND BIASED SAMPLING 


We have found that the additional forces accelerating 
relaxation are derived in the framework of the variational 
principle. The cumulant generating function of the total 
heat in the original system, which has the trivial zero first 
cumulant, is given by the expectation of the housekeep¬ 
ing heat in the biased system. Furthermore, Eq. m 
together with Eq. (1251) implies that the large deviation 
function of the total heat in the original system is given 
by the expectation of the housekeeping heat in the biased 
system: 


Io(q) = q 


By choosing the exponential force m as the addi¬ 
tional force, the dynamics m should be solved under a 


(28) 
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with 


q = 



(29) 


Note that, since (^QhkJ > 0, the large deviation func¬ 
tion Io(q) in Eq. ([28]) only for non-negative q can be 
obtained by measuring the housekeeping heat in the bi¬ 
ased system. The left-hand side of Eq. (l28l) denotes the 
intrinsic fluctuation of the time derivative of the total 
heat, which equals the excess heat, in the original sys¬ 
tem, namely an equilibrium state. On the other hand, 
the right-hand side with Eq. (1^1) is the response of the 
total heat, which equals the expectation of the house¬ 
keeping heat, in the vicinity of the additional nonconser¬ 
vative force u 7 , the biased system. Thus Eq. (1281) can be 
regarded as the exact form of the fluctuation-response re¬ 
lation in the NESS for the case that the stationary distri¬ 
bution is shared with equilibrium system. Furthermore, 
since q represents the energy dissipation under the per¬ 
turbation u 7 , Eq. (l28l) is the exact extension of the FDT 
in the NESS for the case that the stationary distribution 
is fixed. This is the main result of the present paper. 
We here emphasize that the derivation of Eq. (l28l) does 
not resort to a linear approximation, which usually ap¬ 
pears in extensions of the FDT and fluctuation-response 
relation in the NESS. 

We finally remark the role of the additional force in the 
context of the so-called biased samplings before address¬ 
ing the conclusion. Let us compare the path probabilities 
with and without the additional force u 7 as 


V’ut(XIxo) = In 


E u t (X|x 0 ) 
Lo(X|x 0 ) 



u 7 • (x + grad U) 
2 T 


(u 7 ) 1 2 3 4 

4 T 


divu 7 

2 


.(30) 


For both choices of the rotational and exponential forces, 
we find (V'u7) u7 > 0 and (V'u'^o < 0- The first inequal¬ 
ity represents the non-negativity of the KL divergence 
(E^l) and thus holds even if u 7 does not coincide with 
our choices. On the other hand, the nonpositivity in the 
second inequality is in debt to u 7 of our proposal. Since 
the expectation is realized by typical paths in 

the biased system with u 7 , these two inequalities with 
the definition of be., L u -, = exp('0 u7 )Lo indicate 


that switches between the typical and rare path realiza¬ 
tions are induced by the addition of our forces. Thus, 
if the typical path in the system with the DBC is the 
bottleneck of relaxation, such as trap at local minimum, 
the significant reduction of relaxation time is expected 
by our forces EEt The switches between the typical 
and rare path realizations, which governs the relaxation, 
enable us to connect the fluctuation as a rare event rep¬ 
resented by the tail of a large deviation function in the 
original system with the response expressed as a typical 
event in the biased system. 

VIII. CONCLUSION 

We have derived the full order expression of the ex¬ 
tension of the FDT in the NESS under a perturbation 
that leaves the stationary distribution unchanged. The 
obtained FDT has focused on the intrinsic fluctuations in 
an equilibrium state, while the conventional extensions of 
the FDT in the NESS have been focused on those in the 
NESS. The intrinsic fluctuation referred in our FDT is 
expressed in terms of the large deviation of the total heat 
in an equilibrium state. In addition, the conventional ex¬ 
tensions of the FDT in the NESS refers to the “violation 
of the FDT,” in which they regard the response as the 
energy dissipation via a linear response theory and the 
housekeeping heat as the violating term of the FDT. On 
the other hand, our FDT assumes that the response to 
a nonconservative force itself is the housekeeping heat, 
i.e., the energy dissipation. From this viewpoint, the 
relation between the response and energy dissipation is 
straightforwardly given. Therefore only the fluctuation- 
response relation C51) is required to obtain the exten¬ 
sion of the FDT in our case. Our framework may imply 
that physical quantities in the NESS should be observed 
by measuring the difference from an equilibrium state. 
This viewpoint will allow us to give deeper understand¬ 
ings of thermodynamics and statistical physics in non¬ 
equilibrium states. 


ACKNOWLEDGMENTS 

M.O. acknowledges the financial support by MEXT 
in Japan, Grant-in Aid for Young Scientists (B) No. 
24740263, Grant-in-Aid for Scientific Research (B) KAK- 
ENHI No. 15H03699, and the Kayamori Foundation of 
Informational Science Advancement. 


[1] R. Kubo, M. Toda, and N. Hashitsume, Statistical 
Physics II, 2nd ed. (Springer-Verlag, Berlin, 1991). 

[2] J. Prost, J. F. Joanny, and J. M. R. Parrondo, Phys. Rev. 
Lett, 103, 090601 (2009). 

[3] M. Baiesi, C. Maes, and B. Wynants, Phys. Rev. Lett. 
103, 010602 (2009). 

[4] U. Seifert and T. Speck, Europhys. Lett. 89 10007 (2010). 


[5] T. Harada and S.-I. Sasa, Phys. Rev. Lett. 95, 130602 
(2005). 

[6] T. Harada and S.-I. Sasa, Phys. Rev. E 73, 026131 
(2006). 

[7] L. Onsager, Phys. Rev. 37, 405 (1931). 

[8] L. Onsager, Phys. Rev. 38, 2265 (1931). 

[9] A. Ichiki and M. Ohzeki, Phys. Rev. E 88, 020101 (2013). 









6 


[10] M. Ohzeki and A. Ichiki, arXiv: 1307.0434 

[11] M. Ohzeki and A. Ichiki, arXiv: 1503.02356 

[12] T. Nemoto and S.-I. Sasa, Phys. Rev. E 83, 030105 

( 2011 ). 

[13] Y. Sughiyama and M. Ohzeki, J. Stat. Mech. 2013, 
P04012 (2013). 

[14] Y. Sughiyama and M. Ohzeki, Phys. E (Amster¬ 
dam,Neth.) 43, 790 (2011). 

[15] T. Sagawa and H. Hayakawa, Phys. Rev. E 84, 051110 

( 2011 ). 

[16] T. Hatano and S.-I. Sasa, Phys. Rev.Lett. 86, 3463 

( 2001 ). 

[17] R. H. Swendsen and J.-S. Wang, Phys. Rev. Lett. 58, 86 
(1987). 


[181 K. Hukushima and K. Nemoto, J. Phys. Soc. Jpn. 65, 
1604 (1996). 

[19] R. Neal, Statistics Comput. 11 , 125 (2001). 

[20] M. Ohzeki and H. Nishimori, J. Phys. Soc. Jpn. 79, 
084003 (2010). 

[21] M. Ohzeki, Phys. Rev. Lett. 105, 050401 (2010). 

[22] H. Suwa and S. Todo, Phys. Rev. Lett. 105, 120603 

( 2010 ). 

[23] K. S. Turitsyn, M. Chertkov, and M. Vucelja, Phys. D 
(Amsterdam, Neth.) 240, 410 (2011). 

[24] H. C. Fernandes and M. Weigel, Comp. Physics Commun. 
182, 1856 (2011). 

[25] Y. Sakai and K. Hukushima, J. Phys. Soc. Jpn. 82, 
064003 (2013). 


